13 research outputs found
Learning Representations on the Unit Sphere: Application to Online Continual Learning
We use the maximum a posteriori estimation principle for learning
representations distributed on the unit sphere. We derive loss functions for
the von Mises-Fisher distribution and the angular Gaussian distribution, both
designed for modeling symmetric directional data. A noteworthy feature of our
approach is that the learned representations are pushed toward fixed
directions, allowing for a learning strategy that is resilient to data drift.
This makes it suitable for online continual learning, which is the problem of
training neural networks on a continuous data stream, where multiple
classification tasks are presented sequentially so that data from past tasks
are no longer accessible, and data from the current task can be seen only once.
To address this challenging scenario, we propose a memory-based representation
learning technique equipped with our new loss functions. Our approach does not
require negative data or knowledge of task boundaries and performs well with
smaller batch sizes while being computationally efficient. We demonstrate with
extensive experiments that the proposed method outperforms the current
state-of-the-art methods on both standard evaluation scenarios and realistic
scenarios with blurry task boundaries. For reproducibility, we use the same
training pipeline for every compared method and share the code at
https://t.ly/SQTj.Comment: 16 pages, 4 figures, under revie
Contrastive Learning for Online Semi-Supervised General Continual Learning
We study Online Continual Learning with missing labels and propose SemiCon, a
new contrastive loss designed for partly labeled data. We demonstrate its
efficiency by devising a memory-based method trained on an unlabeled data
stream, where every data added to memory is labeled using an oracle. Our
approach outperforms existing semi-supervised methods when few labels are
available, and obtain similar results to state-of-the-art supervised methods
while using only 2.6% of labels on Split-CIFAR10 and 10% of labels on
Split-CIFAR100.Comment: Accepted at ICIP'2
Domain-Aware Augmentations for Unsupervised Online General Continual Learning
Continual Learning has been challenging, especially when dealing with
unsupervised scenarios such as Unsupervised Online General Continual Learning
(UOGCL), where the learning agent has no prior knowledge of class boundaries or
task change information. While previous research has focused on reducing
forgetting in supervised setups, recent studies have shown that self-supervised
learners are more resilient to forgetting. This paper proposes a novel approach
that enhances memory usage for contrastive learning in UOGCL by defining and
using stream-dependent data augmentations together with some implementation
tricks. Our proposed method is simple yet effective, achieves state-of-the-art
results compared to other unsupervised approaches in all considered setups, and
reduces the gap between supervised and unsupervised continual learning. Our
domain-aware augmentation procedure can be adapted to other replay-based
methods, making it a promising strategy for continual learning.Comment: Accepted to BMVC'2
New metrics for analyzing continual learners
Deep neural networks have shown remarkable performance when trained on
independent and identically distributed data from a fixed set of classes.
However, in real-world scenarios, it can be desirable to train models on a
continuous stream of data where multiple classification tasks are presented
sequentially. This scenario, known as Continual Learning (CL) poses challenges
to standard learning algorithms which struggle to maintain knowledge of old
tasks while learning new ones. This stability-plasticity dilemma remains
central to CL and multiple metrics have been proposed to adequately measure
stability and plasticity separately. However, none considers the increasing
difficulty of the classification task, which inherently results in performance
loss for any model. In that sense, we analyze some limitations of current
metrics and identify the presence of setup-induced forgetting. Therefore, we
propose new metrics that account for the task's increasing difficulty. Through
experiments on benchmark datasets, we demonstrate that our proposed metrics can
provide new insights into the stability-plasticity trade-off achieved by models
in the continual learning environment.Comment: 6 pages, presented at MIRU 202
Altimetry for the future: Building on 25 years of progress
In 2018 we celebrated 25 years of development of radar altimetry, and the progress achieved by this methodology in the fields of global and coastal oceanography, hydrology, geodesy and cryospheric sciences. Many symbolic major events have celebrated these developments, e.g., in Venice, Italy, the 15th (2006) and 20th (2012) years of progress and more recently, in 2018, in Ponta Delgada, Portugal, 25 Years of Progress in Radar Altimetry. On this latter occasion it was decided to collect contributions of scientists, engineers and managers involved in the worldwide altimetry community to depict the state of altimetry and propose recommendations for the altimetry of the future. This paper summarizes contributions and recommendations that were collected and provides guidance for future mission design, research activities, and sustainable operational radar altimetry data exploitation. Recommendations provided are fundamental for optimizing further scientific and operational advances of oceanographic observations by altimetry, including requirements for spatial and temporal resolution of altimetric measurements, their accuracy and continuity. There are also new challenges and new openings mentioned in the paper that are particularly crucial for observations at higher latitudes, for coastal oceanography, for cryospheric studies and for hydrology. The paper starts with a general introduction followed by a section on Earth System Science including Ocean Dynamics, Sea Level, the Coastal Ocean, Hydrology, the Cryosphere and Polar Oceans and the ââGreenâ Ocean, extending the frontier from biogeochemistry to marine ecology. Applications are described in a subsequent section, which covers Operational Oceanography, Weather, Hurricane Wave and Wind Forecasting, Climate projection. Instrumentsâ development and satellite missionsâ evolutions are described in a fourth section. A fifth section covers the key observations that altimeters provide and their potential complements, from other Earth observation measurements to in situ data. Section 6 identifies the data and methods and provides some accuracy and resolution requirements for the wet tropospheric correction, the orbit and other geodetic requirements, the Mean Sea Surface, Geoid and Mean Dynamic Topography, Calibration and Validation, data accuracy, data access and handling (including the DUACS system). Section 7 brings a transversal view on scales, integration, artificial intelligence, and capacity building (education and training). Section 8 reviews the programmatic issues followed by a conclusion
Altimetry for the future: building on 25 years of progress
In 2018 we celebrated 25âŻyears of development of radar altimetry, and the progress achieved by this methodology in the fields of global and coastal oceanography, hydrology, geodesy and cryospheric sciences. Many symbolic major events have celebrated these developments, e.g., in Venice, Italy, the 15th (2006) and 20th (2012) years of progress and more recently, in 2018, in Ponta Delgada, Portugal, 25 Years of Progress in Radar Altimetry. On this latter occasion it was decided to collect contributions of scientists, engineers and managers involved in the worldwide altimetry community to depict the state of altimetry and propose recommendations for the altimetry of the future. This paper summarizes contributions and recommendations that were collected and provides guidance for future mission design, research activities, and sustainable operational radar altimetry data exploitation. Recommendations provided are fundamental for optimizing further scientific and operational advances of oceanographic observations by altimetry, including requirements for spatial and temporal resolution of altimetric measurements, their accuracy and continuity. There are also new challenges and new openings mentioned in the paper that are particularly crucial for observations at higher latitudes, for coastal oceanography, for cryospheric studies and for hydrology.
The paper starts with a general introduction followed by a section on Earth System Science including Ocean Dynamics, Sea Level, the Coastal Ocean, Hydrology, the Cryosphere and Polar Oceans and the âGreenâ Ocean, extending the frontier from biogeochemistry to marine ecology. Applications are described in a subsequent section, which covers Operational Oceanography, Weather, Hurricane Wave and Wind Forecasting, Climate projection. Instrumentsâ development and satellite missionsâ evolutions are described in a fourth section. A fifth section covers the key observations that altimeters provide and their potential complements, from other Earth observation measurements to in situ data. Section 6 identifies the data and methods and provides some accuracy and resolution requirements for the wet tropospheric correction, the orbit and other geodetic requirements, the Mean Sea Surface, Geoid and Mean Dynamic Topography, Calibration and Validation, data accuracy, data access and handling (including the DUACS system). Section 7 brings a transversal view on scales, integration, artificial intelligence, and capacity building (education and training). Section 8 reviews the programmatic issues followed by a conclusion
Précision de l'altimétrie satellitaire radar sur les cours d'eau: Développement d'une méthode standard de quantification de la qualité des produits alti-hydrologiques et applications
Numerous works during the last ïŹfteen years have shown the potential contribution of satellite radar altimetry for the monitoring of water levels of inland water bodies (inner seas, lakes,De nombreux travaux menĂ©s durant les quinze derniĂšres annĂ©es ont permis de montrer le potentiel de l'altimĂ©trie satellitaire radar pour le suivi du niveau des eaux continentales (mers intĂ©rieures, lacs, zones d'inondations et grand
Contrastive Learning for Online Semi-Supervised General Continual Learning
International audienceWe study Online Continual Learning with missing labels and propose SemiCon, a new contrastive loss designed for partly labeled data. We demonstrate its efficiency by devising a memory-based method trained on an unlabeled data stream, where every data added to memory is labeled using an oracle. Our approach outperforms existing semi-supervised methods when few labels are available, and obtain similar results to state-of-the-art supervised methods while using only 2.6% of labels on Split-CIFAR10 and 10% of labels on Split-CIFAR100
Learning Representations on the Unit Sphere: Application to Online Continual Learning
16 pages, 4 figures, under reviewWe use the maximum a posteriori estimation principle for learning representations distributed on the unit sphere. We derive loss functions for the von Mises-Fisher distribution and the angular Gaussian distribution, both designed for modeling symmetric directional data. A noteworthy feature of our approach is that the learned representations are pushed toward fixed directions, allowing for a learning strategy that is resilient to data drift. This makes it suitable for online continual learning, which is the problem of training neural networks on a continuous data stream, where multiple classification tasks are presented sequentially so that data from past tasks are no longer accessible, and data from the current task can be seen only once. To address this challenging scenario, we propose a memory-based representation learning technique equipped with our new loss functions. Our approach does not require negative data or knowledge of task boundaries and performs well with smaller batch sizes while being computationally efficient. We demonstrate with extensive experiments that the proposed method outperforms the current state-of-the-art methods on both standard evaluation scenarios and realistic scenarios with blurry task boundaries. For reproducibility, we use the same training pipeline for every compared method and share the code at https://t.ly/SQTj
Calibration of a propagation model in a large river using satellite altimetry
Satellite altimetry may be used for monitoring large rivers, such as Niger River. Since data samples are sparse in time and accuracy of measurements is limited, an interpolation method is developed in order to get water levels at any time and to adjust observed values,taking account of their limited accuracy. The method uses a flood propagation model dedicated to flood propagation in large African rivers calibrated from one gauge station, used as a reference, and satellite altimetric data provided by Topex or Envisat. It allows capturing the water level behavior at the flood peak even though no measurement was available at that time